The present invention relates to a method for reducing blocking artifacts. The invention is useful for removing blocking artifacts from still pictures or moving pictures that were reconstructed according to any coding scheme that introduce blocking artifacts. The invention provides a robust and picture-content dependent solution for removing the blocking artifact without reducing the quality or sharpness of the processed picture, and may be implemented efficiently in software and in hardware.
In Block-Based video coding (such as the ISO standards MPEG1, MPEG2, MPEG4 and JPEG, and ITU standards H.261 and H.263) each picture in the video sequence is partitioned into blocks of N×N pixels, (specifically 8×8 pixels in the MPEG/JPEG compression family and in H.263/H.261), and each block is then coded independently. When the bits-budget for the encoding is limited, a single block may be assigned with fewer bits than required for its representation (“lossy compression”). Since most popular block-based techniques are Discrete Cosine Transform (DCT) based, it is common that the deprived data is the data related to higher spatial frequency. In the extreme case of Very Low Bit rate (VLB) coding, most of the bits are allocated to the mean pixel value of the block, and only a few are allocated to the higher frequency variations. As a result, the continuity between adjacent blocks is broken. This discontinuity at the block boundaries presents an annoying artifact known as the Blocking Artifact. FIG. 1 illustrates the blocking problem: (A)—an original picture, and (B) a reconstructed (encoded/decoded) picture. The picture in (B) clearly shows the blocking artifact.
Existing methods to reduce this blocking artifact, as described for example in “Method of Removing Blocking Artifacts in a Coding System of a Moving Picture”, U.S. Pat. No. 6,240,135 to Kim et. al., (hereinafter “Kim '135”) is unsatisfactory. Other, equally unsatisfactory prior art methods dealing with the subject include ITU H.263 recommendation Annex J: Deblocking filter mode, U.S. Pat. No. 6,028,967, to Kim, U.S. Pat. No. 6,188,799 to Tan, U.S. Pat. No. 6,151,420 to Wober, U.S. Pat. No. 6,236,764 to Zhou, U.S. Pat. No. 6,215,425 to Andrews, U.S. Pat. No. 5,677,736 to Suzuki, U.S. Pat. No. 5,933,541 to Kutka, and U.S. Pat. No. 5,802,218 to Brailean. The Kim '135 patent, assigned to LG Electronics Inc. (hereinafter “the LG method”) is the most relevant to this disclosure, and is discussed in more detail below.
Kim '13 5 considers 5 pixels from each side of a block boundary. An absolute difference is then calculated between each two neighbors (9 pairs). If the absolute difference is larger than a threshold, a “0” is accumulated to a counter, while if it is smaller than the same threshold, a “I” is accumulated to the counter. The outcome is a number between 0 and 9. This is in effect a kind of an “inverse activity” measure over the boundary and into the depth of the block on each side. This inverse activity type of measure serves to decide between two methods (“modes”) of processing:
I. a Default mode (that will in effect change only the 2 boundary pixels), which is operative when the “inverse activity” is small (e.g. in case of textured data).
II. a DC mode, which is operative when the “inverse activity” is large (the area is more or less smooth, with maybe one edge inside it). In the DC mode there are two options: 1) if the difference between the minimal pixel and the maximal pixel in the region of 4 pixels to the left of the boundary and 4 pixels to the right is large (larger or equal to twice the quantization scale (QS) factor of the treated block) then nothing is done (since the algorithm deduces that the blocking artifact is not severe). However, this may be also the case when the blocking artifact is severe and there is also an edge in the 8 pixels region, so that this lack of action is actually a disadvantage. 2) if the difference is small (smaller than twice the QS factor of the treated block) then the entire region of interest (ROI) of the eight pixels is smoothed with a constant filter (since the algorithm deduces that the blocking artifact is severe).
The disadvantages of the LG method become clear when considering the following examples. In the first example (Case “A”), consider a block with a visible and solid block boundary, having a large single high edge within the block (a very common case in natural pictures divided to 8×8 blocks). In such a case, the LG method chooses the DC mode (II). Within the DC mode option I is chosen and no filtering is performed, i.e. the LG method will not filter the visible block boundary of this block. In another, similar example (Case “B”), consider a block in which the edge is not too high, i.e. it is a mild edge. In this case the LG method will also choose the DC mode, but in this case option 2 is chosen and filtering is performed. That is, the LG method will filter the visible block boundary of this block, but will also filter a real edge (that will affect its neighborhood since the LG method use a 9 taps filter in this case).
In a second example (Case “C”), consider a block with a visible and solid block boundary and two edges (one on each side of the boundary). In this case, the LG method will choose the Default mode (I). In the default mode, the LG method will change the value of the 2 pixels that cross the block boundary. The problem here is that the LG method is not exploiting the possibility to smooth deeper into the block, even if the edges are far away from the boundary (say at ±3 pixels from the boundary).
Yet another fault of the LG algorithm (as disclosed in the LG method) is that it is not robust to changes in the threshold against which the absolute differences are evaluated. There are no numerical values for the threshold in the prior art LG patent above (Kim '13 5), and the examples used above assumed the most logical values, taken from the MPEG4 standard in which the LG algorithm is presented with numerical values. If the threshold is changed to a smaller number, then Case “A” above becomes frequent, and the algorithm will produce poor results. If the threshold is changed to a larger number, then the DC mode (II) becomes irrelevant (it will not be activated). Therefore, the LG method is not robust.
There is thus a widely recognized need for, and it would be highly advantageous to have, a low cost implementation, general use, accurate de-blocking filter aimed at reducing the blocking artifact.